Looking under the Hood of Stochastic Machine Learning Algorithms for Parts of Speech Tagging

The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the National Science Foundation, or the U.S. government. We are grateful to Alex Rudnicky from CMU for providing the training data to us and to Yifen Huang, CMU, for discussing the project with us. Abstract A variety of Natural Language Processing and Information Extraction tasks, such as question answering and named entity recognition, can benefit from precise knowledge about a words " syntactic category or Part of Speech (POS) (Stolz, Tannenbaum et al. 1965; Church 1988; Rabiner 1989). POS taggers are widely used to assign a single best POS to every word in text data, with stochastic approaches achieving accuracy rates of up to 96 to 97 percent (Jurafsky and Martin 2000). When building a POS tagger, human beings needs to make a set of decisions, some of which significantly impact the accuracy and other performance aspects of the resulting engine. In this paper we provide an overview of these decisions and empirically determine their impact on POS tagging accuracy. We envision the gained insights to be a valuable contribution for people who want to design, implement, modify, fine-tune, integrate, or simple reasonably use a POS tagger. Based on the results presented herein we built and integrated a POS tagger into AutoMap, a tool that facilitates Natural Language Processing and relational text analysis, as a stand-alone feature as well as an auxiliary for other tasks.